SG-WRAP: A Schema-Guided Wrapper Generator

With the development of the Internet, the World-WideWeb has become everyone’s invaluable information source. However, most of data on the Web is currently in the form of HTML pages, which is neither well-structured nor associated with schema. It is almost impossible to use such data efficiently. Web wrapper technology has been developed to transform unstructured /semi-structured data to semi-structured/structured data, which can be queried and analyzed using matured techniques developed in database and other fields. The main issues of wrapper generation include (1) to identify semantics of the data contained in an HTML document and, (2) to establish the mappings between its structure and its semantics. Various wrapper generation tools developed addressed these issues in different ways [1, 2, 3, 5, 4]. In this paper, we present a wrapper generator, SGWRAP. It was design and developed based on the following observations. First, while user interaction is probably the best way to help the generation of wrappers for a specific HTML source quickly and accurately because of the diversity of HTML pages and limited semantic information expressed in HTML tags, user’s efforts should be minimized as much as possible. Second, the ultimate goal for wrapper generation is to transform the original data into some structured one that is easy to consume, rather than to understand the structure of the original data. When a user gathers data from the Web, s/he must have her/his needs in her/his mind. It is often not necessary to generate wrappers for the entire HTML document. Therefore, SG-WRAP, adopts a novel, schema guided, approach for wrapper generation. With this approach, a user defines the schema of